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FOREWORD 


The  Navy  3ob  Performance  Measurement  (3PM)  Program  is  a  product  of  the  Navy 
Performance-Based  Personnel  Classification  Project.  Both  efforts  constitute  significant 
contributions  to  the  Joint-Service  3PM  Program,  which  can  be  considered  a  landmark 
research  thrust  of  the  armed  services.  This  research  has  been  funded  through  P.E. 
63707N,  Z  1770.001,  and  reimbursable  funds. 


This  report  outlines  the  genesis  and  development  of  the  Navy  3PM  Program  and 
details  the  current  status  and  planned  efforts  of  that  program.  Information  contained 
herein  is  intended  to  benefit  both  the  research  and  the  operational  communities. 
Ultimately,  the  information  generated  by  the  project  can  be  expected  to  benefit  the 
armed  services,  military  as  well  as  civilian  research  communities,  and  applied 
industrial/organizational  psychology  in  general. 


B.  E.  BACON 
Captain,  U.S.  Navy 
Commanding  Officer 


3.  S.  McMICHAEL,  Ph.D. 
Technical  Director 
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SUMMARY 


Problem 

The  Navy,  along  with  the  other  armed  services,  historically  has  not  used  a  prediction 
of  job  performance  to  guide  classification  and  assignment  of  recruits,  even  though  on-the- 
job  performance  is  the  preferred  criterion  for  setting  enlistment  standards.  Instead,  the 
services  rely  primarily  on  a  prediction  of  success  in  entry-level  training  because  training 
grades  are  easy  to  obtain.  This  problem  is  being  addressed  in  the  congressionally 
mandated  Joint-Service  Job  Performance  Measurement  (JPM)/Enlistment  Standards 
Project.  The  Project  is  investigating  the  feasibility  of  measuring  job  performance  in  a 
variety  of  ways  and  demonstrating  the  usefulness  of  this  type  of  criterion  information  in 
setting  enlistment  standards. 

Objective 

The  main  goals  of  the  Joint-Service  JPM/Enlistment  Standards  Project  are  to 
measure  job  performance  and  to  link  enlistment  standards  to  performance  data.  The  Navy 
has  begun  research  in  several  areas  to  support  these  goals.  This  report  documents  and 
summarizes  the  background  developments  that  have  contributed  to  the  inception  of  the 
Navy's  JPM  program. 

Synopsis 


The  Joint-Service  Project  had  its  genesis  in  congressional  concerns  that  the  Armed 
Services  Vocational  Aptitude  Battery  (ASVAB)  was  not  being  validated  against  eventual 
job  performance.  The  project  will  develop  both  hands-on  and  surrogate  measures  of  job 
performance.  This  research  strategy  has  been  endorsed  by  the  Manpower  Accession 
Policy  Steering  Committee  and  the  National  Academy  of  Sciences  (NAS)  Committee  on 
the  Performance  of  Military  Personnel.  At  the  present  time,  research  is  being  conducted 
in  32  occupational  specialty  areas  across  the  four  services.  This  constitutes  about  20 
percent  of  the  total  enlisted  Armed  Forces. 

The  Navy's  contribution  to  the  Joint-Service  Project  is  aimed  at  improving  classifica¬ 
tion  and  assignment  through  the  measurement  of  job  performance.  The  initial  Navy- 
planned  research  efforts  were  truncated  due  to  budgetary  constraints.  Early  Navy  efforts 
focused  primarily  on  measuring  job  performance  for  a  small  set  of  ratings.  In  subsequent 
years,  the  Navy  program  has  been  expanded  in  level  of  research  funding  and  broadened  in 
scope. 

The  current  program  will  cover  seven  ratings  that  fit  well  in  the  Joint-Service  effort 
because  together  they  cover  about  25  percent  of  the  total  Navy  enlisted  force,  come  from 
the  top  26  critical  Navy  ratings,  represent  5  of  the  24  officially  recognized  occupational 
fields,  and  use  5  of  the  current  10  ASVAB  composites  that  are  in  operation  in  the  Navy's 
classification  and  assignment  system.  In  addition,  two  original  research  tasks  (i.e., 
development  of  a  prototype  JPM  data  base  and  demonstration  of  new  computerized 
predictor  tests)  have  become  separate  projects.  If  fully  developed,  the  JPM  data  base  can 
be  used  by  the  research  community  to  guide  criterion  development  and  by  the  operational 
community  to  answer  policy  questions  and  guide  decisions.  The  work  on  new  predictors 
was  to  be  conducted  late  in  the  life  of  the  project,  but  new  test  development  technology 
is  progressing  at  a  very  fast  rate.  With  the  impending  advent  of  the  computerized 
ASVAB,  there  is  an  increasing  amount  of  interest  in  evaluating  new  predictors  to 
supplement  this  selection  battery. 


This  report  covers  the  history  of  the  3PM  Program  and  describes  the  current  scope  of 
the  program  as  well  as  plans  for  the  future  use  of  3PM  technology. 

The  3PM  Program,  if  successful,  will  provide  the  Navy  with  the  capability  to  measure 
and  predict  job  performance.  As  noted  by  the  NAS  Committee,  "If  the  services  can 
develop  substitutes  from  the  same  research  base  that  compare  favorably  with  hands-on 
measures,  they  will  have  criterion  measures  of  continuing  usefulness  and  will  be  able  to 
set  up  a  program  of  ongoing  validity  research"  (Office  of  the  Assistant  Secretary  of 
Defense;  Manpower,  Installations,  and  Logistics;  1984,  p.  24).  Such  a  program  in  the  Navy 
will  lead  to  major  improvements  in  personnel  selection  and  other  personnel  system 
functions  and  significant  contributions  to  measurement  technology  in  particular  and 
applied  psychology  in  general. 
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INTRODUCTION 


The  Armed  Forces  need  accurate  enlistment  screening  and  job  placement  to  ensure 
training  success,  retention  of  skilled  personnel,  and  mission  performance.  Suboptimal 
assignment  of  personnel  to  jobs  can  adversely  impact  readiness  and  result  in  critical  skill 
shortages.  High  training  costs,  dissatisfied  personnel  and  decreases  in  morale,  pro¬ 
ductivity,  and  retention  can  frequently  be  traced  to  deficiencies  in  classification  and 
assignment. 

Problem 

The  general  problem  is  that  the  Navy,  along  with  the  other  armed  services,  has  never 
used  predictions  of  job  performance  to  guide  classification  and  assignment  of  recruits. 
This  has  not  been  done  despite  the  fact  that  on-the-job  performance  is  generally 
recognized  as  the  best  criterion  for  validating  selection  instruments  and  setting  enlist¬ 
ment  standards.  Instead,  we  rely  primarily  on  predictions  of  success  in  entry-level 
training,  because  this  criterion  measure  is  easy  to  obtain.  The  feasibility  of  measuring  job 
performance  and  using  this  type  of  criterion  information  to  set  enlistment  standards  is 
being  investigated  in  a  Joint-Service  Job  Performance  Measurement  (JPM)/Enlistment 
Standards  Project. 

Objective 

The  main  goals  of  the  Joint-Service  JPM/Enlistment  Standards  Project  are  to 
measure  job  performance  and  to  link  enlistment  standards  to  performance  data.  The  Navy 
has  begun  research  in  several  areas  to  support  these  goals.  This  report  summarizes  the 
background  developments  that  have  contributed  to  the  Navy's  JPM  program. 

Background 

Aptitude  tests  have  been  used  to  determine  military  enlistment  since  World  War  I, 
when  the  Army  Alpha  test  was  developed.  As  the  state  of  the  art  of  personnel  research 
progressed,  so  did  military  aptitude  tests.  The  latest  selection  instrument,  the  Armed 
Services  Vocational  Aptitude  Battery  (ASVAB),  has  been  in  use  in  various  forms  since 
1976.  All  of  the  services  use  the  ASVAB  to  assess  applicant  aptitudes  and  to  classify  and 
assign  new  recruits  to  occupational  specialties  (called  ratings  in  the  Navy). 


OBJECTIVES  OF  JOINT-SERVICE  PROJECT 

•  Develop  prototype  methodologies  for 
measuring  job  performance 

>  Link  enlistment  standards  to  on-the-job 
performance 


The  ASVAB  is  a  paper-and-pencil  te^t  of  cognitive  abilities  that  has  been  proven  to 
predict  training  success  but  not  on-the-job  performance.  It  is  widely  recognized, 
however,  that  job  performance  should  be  the  ultimate  basis  for  validating  selection 
instruments  and  setting  enlistment  standards. 

Historically,  training  performance  has  been  used  to  validate  enlistment  and  classifi¬ 
cation  standards  because  it  appears  to  be  a  logical  surrogate  for  later  job  performance. 
More  importantly,  training  data  are  economical  to  collect  and  analyze.  Training  grades 
are  based,  for  the  most  part,  on  paper-and-pencil  tests  that  do  not  assess  job  skills. 
Moreover,  training  success,  however,  will  always  remain  an  inadequate  validation 
criterion  to  the  extent  that  it  does  not  mirror  the  actual  job  environment  or  include  all  of 
the  tasks  performed  in  a  job. 

Validation  of  selection  instruments  by  job  performance  data  was  not  feasible  until 
recent  years  for  several  reasons:  (1)  Hands-on  performance  tests  for  most  military  jobs 
were  unavailable:  (2)  the  acquisition  and  manipulation  of  required  job-task  information 
was  prohibited  by  a  lack  of  computer  resources;  and  (3)  the  development,  administration, 
and  validation  of  job  performance  measures  were  fiscally  unrealistic  (Office  of  the 
Assistant  Secretary  of  Defense;  Manpower,  Reserve  Affairs,  and  Logistics  (OASD, 
MRA&L),  1982a). 

Validation  using  job  performance  data  became  feasible  only  recently  due  to  advances 
in  personnel  research  and  computer  technology.  While  test  administration  costs  remain 
high,  some  developmental  expenses  are  lower  due  to  the  existence  of  job  analysis 
information  for  most  occupations  in  the  armed  services.  With  a  Joint-Service  research 
strategy  aimed  at  developing  empirical  substitutes  for  hands-on  performance  tests,  it  is 
now  possible  to  undertake  a  Department  of  Defense  (DoD)  project  to  link  enlistment 
standards  with  job  performance.  The  Joint-Service  Project  represents  the  first  research 
effort  aimed  at  validating  ASVAB  against  actual  job  performance. 

This  report  is  organized  into  four  sections.  The  first  describes  the  evolution  and 
research  strategy  of  the  Joint-Service  JPM/Enlistment  Standards  Project.  The  second 
covers  the  initiation  of  the  Navy's  JPM  research  efforts.  The  third  section  details  the 
Navy's  advanced  development  effort  entitled  "Performance-Based  Personnel  Classifica¬ 
tion."  Finally,  the  fourth  section  describes  the  Navy's  JPM  program,  future  plans  for  JPM 
technology,  and  the  anticipated  end-products  of  the  Navy  research.  Figure  1  provides  an 
overview  of  the  history  and  development  of  the  Navy’s  JPM  program. 


Performance-Based  Personnel  Classification  Project  expands 
and  evolves  into  wider  ranging  JPM  program. 


1985 


I  Navy's  Performance-Based  Personnel  Classification  Project  is 

initiated. 

1984  NAS  endorses  goals  and  objectives  of  Joint-Service  Project. 

MAP  Steering  Committee  endorses  Joint-Service  research 
strategy. 

1983  Exploratory  development  funds  are  reprogrammed  to  support 
additional  planning  for  JPM  research. 

JPM  Working  Group  is  established. 

Joint-Service  Project  is  formally  launched. 

1982  House  Appropriations  Committee  tasks  OASD  to  oversee  Joint 

j  Service  Project. 

CLASP  is  implemented 

1981  House  Armed  Services  Committee  endorses  Joint-Service 

Project. 

Joint-Service  Job  Performance  Measurement/Enlistment 
Standards  Project  is  introduced. 

Navy  pilot  studies  are  started. 

OASD  suggests  validation  of  enlistment  standards  against  job 
performance. 

iy80  Congress  raises  concerns  about  the  adequacy  of  selection 
|  |  standards. 

1976  ASVAB  is  implemented. 

I  I 

1973  PRIDE  is  introduced  at  Military  Entrance  Processing  Stations. 

Training  performance  first  used  to  validate  enlistment  and 
classification  standards. 

1918  Army  Alpha  isdeveloped. 

U 


Figure  1 .  The  development  of  the  job  performance  measurement  program 
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THE  JOINT-SERVICE  PROJECT 


The  Joint-Service  JPM/Enlistment  Standards  Project  was  brought  about  by  several 
coinciding  factors.  The  Conference  on  Joint-Service  Efforts  to  Link  Enlistment  Standards 
and  Job  Performance  (OASD,  MRA&L,  1982a)  identified  the  following  major  influences: 
(1)  doubts  about  the  adequacy  of  the  selection  process,  (2)  recognition  that  the  supply  of 
eligible  enlistees  is  dwindling,  (3)  perception  of  a  decrease  in  the  quality  of  accessions,  (4) 
concern  about  opportunities  to  increase  and  maintain  manpower  quality,  and  (5)  advances 
in  testing  and  performance  measurement  technologies. 

Evolution  of  the  Joint-Service  Project 


The  Joint-Service  Project  had  its  origin  in  a  1980  memorandum  from  the  OASD  that 
identified  the  need  to  measure  on-the-job  performance  and  asked  the  services  to  validate 
enlistment  standards  against  such  performance  (App.  A  contains  this  memorandum).  Each 
service  initiated  pilot  research  in  response  to  this  request.  In  1981,  the  House  Armed 
Services  Committee  tasked  the  OASD  to  oversee  a  Joint-Service  Project  in  the  JPM  area 
and  to  detail  progress  in  an  annual  report. 

The  Joint-Service  JPM/Enlistment  Standards  Project  was  formally  launched  at  a 
conference  at  the  Air  Force  Human  Resources  Laboratory,  Brooks  Air  Force  Base,  Texas, 
in  September  1982.  The  objectives  of  this  DoD  conference  were  to 

1.  Fully  elaborate  the  concept,  objective,  and  status  of  the  Joint-Service  JPM 
Project. 

2.  Present  and  discuss  the  proposed  conceptual  framework  for  this  long-term 
project. 

3.  Review  the  status  of  relevant  Army,  Navy,  Marine  Corps,  and  Air  Force  research 
and  development  activities.  (OASD,  MRA&L,  1982a) 

As  a  result,  the  Joint-Service  JPM  Working  Group  was  established  in  October  1982. 
It  is  composed  of  technical  and  a  policy  representatives  from  each  service  and  is  chaired 
by  a  DoD  Joint-Service  Project  Manager.  Its  charter  is  to  review  and  coordinate  the  JPM 
research  programs.  As  part  of  its  mission,  the  Working  Group  shares  research  results  and 
ideas,  thereby  eliminating  unnecessary  redundancy  across  service  research  programs.  The 
group  also  makes  recommendations  on  accession  issues  and  ensures  that  service  data 
collection  efforts  will  obtain  the  information  needed  to  link  enlistment  standards  to  job 
performance. 

Management  Oversight 


An  Annual  Report  to  the  House  Committee  on  Appropriations,  which  details  the 
ongoing  JPM  research,  is  the  main  document  used  by  Congress  to  track  progress.  The 
JPM  Working  Group  prepares  this  report  and  serves  as  the  point  of  contact  for  two  other 
oversight  groups. 


JOINT-SERVICE  OVERSIGHT 


•  JPM  WORKING  GROUP 

Members:  Technical  and  policy  representatives 
from  each  service 

Chair:  DoD  Joint-Service  Project  manager 


MAP  STEERING  COMMITTEE 

Members:  Senior  military  officers  in  personnel 

area  and  commander  of  MEPCOM 

Chair:  Director  for  Accession  Policy 

(OASD/MI&L) 


•  NAS  COMMITTEE  ON  THE  PERFORMANCE  OF 
MILITARY  PERSONNEL 
Members:  Nationally  recognized  experts  in 
performance  measurement  and  selection 
standards 

Chair:  DoD  Manager  (OASD/MI&L) 


The  Manpower  Accession  Policy  (MAP)  Steering  Committee  (formerly  the  ASVAB 
Steering  Committee)  is  the  major  source  of  policy  oversight  for  the  Joint-Service  Project. 
The  steering  committee  is  chaired  by  the  Director  for  Accession  Policy  (OASD; 
Manpower,  Installations,  and  Logistics;  MI&L)  and  is  composed  of  senior  military  officers 
in  the  personnel  area  and  the  commander  of  the  military  enlisted  processing  command 
(MEPCOM).  The  MAP  Steering  Committee  reviews  research  progress  from  a  manpower, 
personnel,  and  training  perspective  and  ensures  coordination  of  research  and  policy  issues. 


The  National  Academy  of  Sciences  (NAS)  Committee  on  the  Performance  of  Military 
Personnel  is  composed  of  nationally  recognized  experts  in  scientific  and  technical  areas 
related  to  performance  measurement  and  selection  standards,  including  personnel  testing, 
industrial  psychology,  research  design,  performance  evaluation,  military  occupational 
analysis,  et  cetera.  The  committee  provides  independent  technical  review  of  the  Joint- 
Service  Project,  assisting  DoD  and  the  services  in  determining  data  requirements  and 
strategies  for  linking  enlistment  standards  to  job  performance. 


Overall  Research  Strate* 


During  its  first  year,  the  JPM  Working  Group  planned  the  Project  research  program. 
In  broad  terms,  the  plan  calls  for  the  systematic  identification  of  a  set  of  occupational 
specialties  to  serve  as  a  test  bed  for  investigating  hands-on  performance  tests  and 
selected  surrogate,  less  expensive  measurement  instruments.  Performance  data  are  to  be 


related  to  ASVAB  scores  as  part  of  a  demonstration  of  the  use  of  this  type  of  criterion 
data,  setting  the  stage  for  linking  enlistment  standards  to  job  performance.  This  strategy 
was  endorsed  by  the  MAP  Steering  Committee  during  the  summer  of  1983  (App.  B). 
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JOINT-SERVICE  RESEARCH  STRATEGY 


•  Select  occupational  specialties  as  a  Joint-Service 
test  bed. 

•  Develop  hands-on  performance  measures. 

•  Develop  surrogate  measures  to  compare  with 
hands-on  measures. 

•  Analyze  data  to  determine  whether  surrogate 
measures  can  be  substituted  for  hands-on 
measures. 

•  Attempt  to  link  performance  measures  with 
ASVAB. 


The  Joint-Service  research  strategy  was  reviewed  by  the  NAS  committee  and  the 
JPM  Working  Group  at  a  workshop  held  at  the  end  of  1983  (OASD,  MRA&L,  1983).  The 
specific  objectives  of  the  Joint-Service  Project,  as  stated  at  the  JPM  workshop,  were  (1) 
to  develop  prototypic  methods  for  measuring  job  performance;  and  (2)  if  possible,  to  link 
enlistment  standards  to  on-the-job  performance.  Strategies  for  linking  enlistment 
standards  to  job  performance  were  not  specified.  The  JPM  Working  Group  assumed  that 
the  best  way  to  structure  the  Joint-Service  effort  was  to  have  each  service  conduct  its 
own  research,  according  to  and  in  concert  with  Joint-Service  guidelines. 

Selection  of  Occupational  Specialties 

One  of  the  first  guidelines  generated  by  the  working  group  was  a  set  of  criteria  for 
all  services  to  use  in  selecting  occupational  specialties  for  their  research  programs.  Six 
were  identified:  (1)  The  occupations  should  be  of  critical  importance  to  each  service;  (2) 
there  should  be  enough  people  assigned  to  them  to  ensure  adequate  sample  size;  (3)  enough 
individuals  in  a  specialty  should  be  located  on  as  few  bases  as  possible  in  order  to  obtain 
an  adequate  sample  without  undue  travel  costs;  (4)  the  important  tasks  within  the  occupa¬ 
tional  specialty  should  be  measurable;  (5)  any  problems  within  the  occupational  specialty 
(e.g.,  attrition)  should  be  known  and  well  documented;  and  (6)  characteristics  of  the 
occupation  should  permit  evaluation  of  the  impact  on  minorities  and  women  of  the 
measurement  techniques  and  data  collection. 
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Each  service  used  these  criteria  to  select  from  6  to  19  occupational  specialties  for 
study,  agreeing  to  collect  a  common  set  of  data  inputs  (e.g.,  ASVAB  scores,  training 
grades,  etc.).  In  addition,  each  service  agreed  to  act  as  the  "lead  service"  in 
demonstrating  job  performance  measures  for  one  cross-service  occupation. 

Development  of  Performance  Measures 

The  JPM  Working  Group  also  agreed  that  the  following  premises  were  central  to  the 
development  of  job  performance  measures:  (1)  The  primary  focus  of  measurement  should 
be  "manifest,  observable  job  behaviors";  (2)  the  most  direct  measures  of  job  behaviors 
should  have  the  highest  likelihood  of  meeting  validity  requirements;  and  (3)  the  hands-on 
job-sample  test  should  have  the  highest  fidelity  to  actual  job  performance  and  will, 
therefore,  serve  as  the  benchmark  to  which  other,  less  direct  measures  will  have  to 
conform. 

The  major  assumption  of  the  Joint-Service  Project  is  that  occupational  analysis  and 
performance  measurement  has  advanced  to  the  point  where  hands-on  performance  tests 
are  feasible  and  can  be  put  to  widespread  use.  The  JPM  Working  Group,  however, 
recognized  that  the  cost  of  such  tests  would  prohibit  DoD-wide  performance  testing  on 
the  large  scale  required  for  validity  research.  Therefore,  they  devised  a  research  strategy 
to  develop  less  expensive,  easier  to  administer  surrogate  measures  that  could  substitute 
for  hands-on  measures.  The  procedure  to  be  applied  was  to  first  develop  a  good  hands-on 
test  as  a  high-fidelity  "benchmark"  against  which  various  surrogate  measures  would  be 
compared. 

Summary 

The  individual  services  are  developing  job  performance  measures  within  a  common 
framework  and  will  ultimately  link  these  measures  to  enlistment  standards.  The 
essentials  of  the  overall  research  strategy  are  (1)  selection  of  an  integrated  set  of  cross¬ 
service  and  service-specific  occupational  specialties  as  a  Joint-Service  test  bed,  (2) 
development  of  hands-on  performance  measures  for  all  of  the  occupational  specialties 
selected,  (3)  development  of  surrogate  measures  to  be  compared  with  the  hands-on 
measures,  and  (4)  analysis  of  the  data  to  examine  the  feasibility  of  substituting  one 
measure  for  another  and  to  establish  relationships  between  the  various  performance 
measures  and  ASVAB. 

In  a  report  on  the  JPM  workshop  (Committee  for  the  Performance  of  Military 
Personnel,  1 984)  and  later,  as  part  of  a  report  to  Congress  (OASD,  MI&L,  1984),  the  NAS 
endorsed  the  goals  and  objectives  of  the  Joint-Service  Project.  The  Committee  also  found 
the  overall  strategy  adopted  by  the  JPM  Working  Group  to  be  reasonable  and  timely  and 
were  optimistic  about  the  potential  of  the  Project  to  improve  the  psychometric  aspects  of 
selection  and  classification: 

The  psychological  profession  has  long  paid  homage  to  the  goal  of 
measuring  performance  directly.  But  hands-on  measures  have  not  been 
used  very  much  in  actual  practice  because  of  the  sheer  difficulty  of 
developing  and  administering  such  measures.  The  Committee  thinks  it 
important  that  this  attempt  is  now  being  made.  .  .  .The  Committee 
supports  the  decision  to  develop  both  hands-on  and  surrogate  measures. 

It  makes  sense  to  commit  enough  financial  and  human  resources  to  give 
the  hands-on  test  a  real  trial.  There  are  few  institutions  in  American 
society  capable  of  doing  so.  At  the  same  time,  the  state  of  testing 


technology  is  such  that  the  hands-on  test  is  likely  to  be  too  unwieldy 
and  expensive  to  use  on  any  continuing  basis.  If  the  Services  can 
develop  substitutes  from  the  same  research  base  that  compare  favor¬ 
ably  with  the  hands-on  measures,  then  they  will  have  criterion  measures 
of  continuing  usefulness  and  will  be  able  to  set  up  a  program  on  ongoing 
validity  research.  (OASD,  MI&L,  1984,  p.  24) 

At  the  present  time,  research  is  focused  on  32  occupational  specialties  across  the 
four  services.  This  constitutes  about  20  percent  of  the  total  enlisted  Armed  Forces.  If 
effective  job  performance  measures  are  developed  for  these  occupations  and  the  link  to 
enlistment  standards  is  accomplished,  the  research  can  be  expanded  to  the  remaining 
occupations.  The  common  framework  adopted  by  the  0PM  Working  Group  "will  provide 
DoD  and  the  Services  with  technology  transfer  benefits  and  considerable  cost  savings, 
while  preserving  Service-specific  performance  and  standards  assessment"  (OASD,  MI&L, 
1984,  p.  5). 


NAVY  PROGRAM  PLANNING  AND  INITIATION 

Background;  PRIDE  and  CLASP 

There  has  been  a  continuing  effort  to  improve  the  classification  and  assignment  tools 
that  are  available  to  Navy  decision  makers.  In  1965,  a  computer-assisted  recruit 
assignment  system  was  developed  to  process  school  recommendations  made  by  classifiers 
at  Recruit  Training  Commands.  In  1973,  a  computerized  school-seat  reservation  network 
named  PRIDE  (personalized  recruitment  for  immediate  and  delayed  enlistment)  was 
introduced  at  what  are  known  as  MEPS  (military  enlisted  processing  stations). 

The  PRIDE  system  operated  on  a  first-come,  first-served  basis  for  each  recruit. 
Analysis  of  the  classification  and  assignment  decisions  (Kroeker  <5 c  Rafacz,  1983)  revealed 
the  following  characteristics:  (1)  Assignment  options  were  not  offered  on  the  basis  of 
predicted  technical  ("A")  school  success;  (2)  ability  levels  of  recruits  were  not  matched 
with  job  complexity;  (3)  changing  Navy  priorities  were  not  accommodated  in  a  timely 
fashion;  (4)  recruits'  preferences  were  not  taken  into  account  in  a  consistent  manner;  (5) 
important  administrative  objectives,  such  as  uniform  minority  and  "A"  school  accession 
rates,  were  not  met;  and  (6)  recruiting  goals  and  openings  in  Navy  ratings  were  given  more 
emphasis  than  the  appropriateness  of  the  rating  for  the  recruit. 

As  a  result,  an  automated  classification  and  assignment  system  called  CLASP 
(classification  and  assignment  within  PRIDE)  was  developed.  This  system  was  imple¬ 
mented  in  1981.  Originally,  it  took  five  factors  (e.g.,  prediction  of  "A"  school  success) 
into  account  in  an  attempt  to  achieve  the  best  person-job  match.  A  sixth  factor 
(prediction  of  attrition)  was  recently  incorporated  into  the  system  (Kroeker  &  Folchi, 
1984).  By  making  this  system  more  performance-based,  it  may  be  possible  to  further 
improve  classification  and  assignment.  (For  a  more  detailed  description  of  the  operation 
of  CLASP,  see  "The  Initial  Navy  Program;  Performance-Based  Personnel  Classification," 
below.) 

Past  Research  Efforts 


Past  research  at  the  Navy  Personnel  Research  and  Development  Center  (NAVPERS- 
RANDCEN)  in  the  performance  measurement  area  have  concentrated  on  assessing 
training  effectiveness  and  productivity.  Several  pilot  studies  initiated  after  the  1980 


OASD  (MRA&L)  memorandum  (see  App.  A)  suggested  that  the  services  investigate  the 
feasibility  of  validating  enlistment  standards  against  job  performance.  Based  on  results  of 
past  research  and  the  pilot  studies,  a  Navy  JPM  program  to  support  personnel  selection 
was  planned.  The  initial  roadmap  was  outlined  at  the  Conference  on  Joint-Service  Efforts 
to  Link  Enlistment  Standards  and  Job  Performance  (OASD,  MRA&L,  1982a). 

NAVPERSRANDCEN  and  its  preceding  organizations  have  actually  been  active 
measuring  performance  on  the  job  over  the  past  two  decades.  Most  of  the  job  sample  or 
simulated  job  tests,  however,  were  developed  in  a  training  context.  For  example,  one 
effort  set  up  a  testing  and  training  program  in  which  hands-on  tests  on  actual  equipment 
were  used  to  diagnose  individual  deficiencies  and  assign  remedial  self-paced  training 
(Anderson,  Laabs,  Pickering,  Sc  Winchell,  1977;  Laabs,  1976;  Laabs,  Main,  Abrams,  Sc 
Steinemann,  1975;  Laabs,  Panell,  Sc  Pickering,  1977;  Winchell,  Panell,  Sc  Pickering,  1976). 

Another  recent  effort  developed  computerized  testing  that  simulated  job  tasks  for 
one  rating,  or  job  (Mackie,  Schultz,  Sc  Bearden,  1981).  The  purpose  of  this  system  was  to 
gather  performance  data  from  individuals  and  provide  feedback  on  group  performance  to 
the  training  commands  so  they  could  revise  their  curriculum. 

Navy  research  on  productivity  has  also  depended  heavily  upon  the  objective  measure¬ 
ment  of  job  performance.  For  example,  the  incentive  programs  that  were  set  up  for 
keypunch  operators  in  naval  shipyards  depended  upon  a  count  of  the  number  of  records 
typed  and  the  number  of  errors  (Nebeker  Sc  Nocella,  1979). 

In  a  recent  review  of  NAVPERSRANDCEN  research  efforts  in  job  performance 
testing,  Pickering  and  Bearden  (1984)  concluded,  among  other  things,  that  (1)  procedures 
for  developing  and  administering  job  performance  tests  needed  improvement,  (2)  not 
enough  is  known  about  the  degree  of  simulation  required  to  measure  the  capability  to 
perform  various  types  of  job  tasks,  and  (3)  there  has  been  a  lack  of  systematic  studies 
demonstrating  the  cost-effectiveness  of  job  performance  tests.  They  also  recommended 
that  job  performance  tests  be  developed  to  validate  ASVAB  composites  for  those  critical 
tasks  where  no  other  adequate  job  performance  measures  exist. 

Pilot  Work 

In  the  1980  memorandum  to  the  service  secretaries  (App.  A),  the  Assistant  Secretary 
of  Defense  (MRA&L)  expressed  the  belief  that  enlistment  and  assignment  standards  should 
be  based  on  the  probability  of  successful  job  performance.  The  large-scale  effort  required 
to  relate  these  standards  to  job  performance  was  seen  as  comprising  three  phases:  (1) 
demonstration  of  the  feasibility  of  the  project,  (2)  validation  of  current  standards  against 
job  performance  indices,  and  (3)  improvement  of  measures  of  potential  ability  and  job 
performance. 


NAVY  PILOT  STUDIES 


•  ASVAB  AND  INDIRECT  JPM  MEASURES 
Objective:  To  validate  ASVAB  against 
surrogate  performance  measures  for  two 
ratings  using  personnel  records 

•  ASVAB  AND  DIRECT  JPM  MEASURES 
Objective:  To  validate  ASVAB  against  direct 
performance  measures  for  two  ratings  using 
existing  test  instruments 

•  LITERATURE  REVIEW 

Objective:  To  review  all  military  research 
between  the  years  1952-1980  dealing  with  job 
performance  prediction 

•  RATING  SELECTION 

Objective:  To  produce  two  sets  of  candidate 
ratings,  based  on  population  size  or  criticality, 
for  use  in  Navy's  JPM  program 


As  part  of  the  first  phase  of  the  DoD  suggested  effort,  the  Navy  conducted  a 
literature  review  and  several  pilot  studies,  some  of  which  attempted  to  validate  ASVAB 
against  indirect  performance  measures  (personnel  records)  and  against  direct  job  perfor¬ 
mance  measures  using  existing  test  instruments.  They  also  included  a  review  of  the 
literature  on  the  prediction  of  military  job  performance.  In  a  final  study,  Navy  ratings 
were  ranked  according  to  different  criteria  to  produce  two  sets  of  candidate  ratings  for 
follow-up  work  in  the  Navy's  3PM  program. 

Literature  Review 

As  part  of  the  pilot  work,  all  military  research  dealing  with  job  performance 
prediction  for  the  period  from  1952  to  1980  was  reviewed  (Vineberg  &  Joyner,  1982).  Job 
knowledge  tests  were  found  to  provide  the  most  practical  method  for  objective  perfor¬ 
mance  measurement  in  the  majority  of  jobs.  They  are  less  expensive  than  job  sample 
tests  and  more  suitable  for  jobs  in  which  incumbents  are  widely  dispersed.  Because  of  the 
high  expense  of  developing  job  sample  tests  for  ASVAB  validation,  the  use  of  such  tests  is 
impractical  except  for  jobs  that  require  expensive  training  programs.  Supervisory  ratings 
of  job  performance  were  viewed  as  dubious  in  value  because  (1)  personal  variances  in 
raters'  judgments  make  the  ratings  unsuitable  as  objective  measures  of  technical 
performance,  and  (2)  supervisors  often  lack  familiarity  with  the  total  job  performance  of 
the  persons  evaluated. 
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ASVAB  and  Indirect  Job  Performance  Measures 


Potentially,  indices  of  job  performance  might  be  found  in  the  enlisted  personnel 
records.  One  study  extracted  a  combined  "advancement  and  survivability"  factor  from 
these  records  to  be  used  as  a  surrogate  measure  of  job  performance  (Simms  <5c  Hiatt, 
1981).  The  objective  was  to  determine  how  Armed  Forces  Qualification  Test  (AFQT) 
scores  relate  to  advancement.  It  was  concluded  that  AFQT  scores  could  be  used  to 
predict  advancement  of  recruits  (especially  those  who  are  not  high  school  graduates)  in 
the  Ship's  Serviceman  rating  but  not  in  the  Electronics  Technician  rating. 

No  strong  conclusions  could  be  drawn  on  the  basis  of  this  limited  study,  so  a  further 
investigation  of  measures  from  personnel  records  was  undertaken.  An  extensive  data  base 
was  assembled,  but  it  was  discovered  subsequently  that  the  performance  variables  could 
not  be  interpreted  unequivocally  as  measures  of  technical  proficiency,  as  required  by  the 
Joint-Service  research  strategy. 

ASVAB  and  Direct  Job  Performance  Measures 

One  of  the  jobs  examined  in  a  pilot  study  was  that  of  the  aviation  submarine  warfare 
operator,  who  performs  detection  and  classification  tasks  using  visually  presented 
acoustic  data.  Performance  data  were  taken  from  a  job  knowledge  test  and  a  job  sample 
test  that  were  part  of  an  existing  evaluation  program.  For  both  job  performance  tests, 
different  ASVAB  composites  were  more  valid  predictors  than  the  current  selector 
composite  that  predicts  training  grades  (Bearden,  198  1). 

In  another  pilot  study,  a  computerized  testing  device  originally  developed  to  assess 
training  effectiveness  was  used  to  test  sonar  technicians  on  several  high-fidelity  simula¬ 
tions  of  job  tasks  in  submarine  detection  and  classification.  Although  the  current  ASVAB 
composite  was  the  most  predictive  of  school  performance,  a  different  composite  was  most 
predictive  of  performance  on  the  simulated  tasks  (Mackie,  Ridihalgh,  Seltzer,  &  Schultz, 
1981). 

The  results  of  both  of  these  pilot  studies  were  tentative  because  of  small  sample 
sizes.  What  both  point  out,  however,  is  that  job  performance  tests  and  perhaps  even  job 
performance  data  may  already  be  available  within  existing  systems  or  data  bases. 

Rating  Selection 

In  anticipation  of  launching  a  full-scale  validation  effort  for  a  large  number  of  Navy 
ratings,  a  final  study  was  completed  to  aid  the  process  of  selecting  the  ratings  to  be  used 
(Nugent,  1981).  Eighty-one  entry-level  ratings  were  identified  and  ranked  from  most  to 
least  populous.  The  ratings  were  also  rank  ordered  for  criticality  to  mission  accomplish¬ 
ment.  These  data  provide  information  for  trade-off  analyses  to  determine  whether 
population  size  or  criticality  of  ratings  should  be  included  in  the  Navy's  JPM  program. 

Initial  Roadmap 

The  roadmap  for  JPM  research  was  formulated  based  on  past  research  and  the  recent 
pilot  work.  Since  the  pilot  studies  did  not  convincingly  demonstrate  the  feasibility  of 
setting  standards  based  on  job  performance,  the  roadmap  included  aspects  of  all  three 
phases  suggested  by  OASD  (i.e.,  feasibility  investigations,  validity  work,  and  measure¬ 
ment). 
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JPM  RESEARCH  ROADMAP 


•  FRONT-END  ANALYSES 

Develop  job  performance  prediction 
information  system. 

Identify  ratings  to  be  covered  in  future  JPM 
research. 

•  MEASUREMENT  EVALUATION 
Construct  and  administer  performance 
measures. 

Develop  improved  methods  for  measuring 
potential  ability  (e  g.,  computerized 
measures). 

•  REVISION  OF  STANDARDS 

Examine  CLASP  for  potential  to  incorporate  a 
prediction  of  future  job  performance. 
Evaluate  impact  of  proposed  changes  to 
CLASP. 
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The  roadmap  provides  a  framework  for  the  systematic  data  gathering  needed  to  make 
the  Navy's  operational  CLASP  more  performance-based.  It  is  organized  around  three  key 
areas:  (1)  front-end  analyses,  (2)  measurement  evaluation,  and  (3)  revision  of  standards. 
Each  area  includes  two  major  research  tasks,  as  described  below. 

Front-End  Analyses 

The  first  major  task  will  develop  an  information  system  to  support  the  prediction  of 
job  performance.  It  follows  directly  from  the  literature  review  that  was  part  of  the  pilot 
work.  It  includes  such  actions  as  the  review  of  existing  data  bases,  including  research  and 
operational  programs,  for  useful  performance  tests  and  data  as  well  as  for  detailed  task 
analysis  information  that  might  be  used  to  build  performance  measures.  All  information 
will  be  catalogued  by  rating.  The  ultimate  objective  is  a  rating-specific  information 
system  that  contains  ail  findings  and  data  applicable  to  relating  selection  and  classifica¬ 
tion  standards  to  job  performance. 

Another  major  task  will  identify  the  ratings  to  be  covered  in  future  JPM  research. 
The  pilot  study  that  identified  candidate  ratings  based  on  size  and  criticality  is  the 
starting  point  for  this  work.  Also  included  are  examinations  of  ways  to  minimize  the 
number  of  ratings  for  which  data  need  to  be  collected  and  to  increase  the  sample  sizes  by 
exploiting  the  task  commonalities  that  exist  across  ratings. 
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Measurement  Evaluation 


A  major  research  task  in  this  key  area  addresses  the  construction  and  administration 
of  performance  measures.  The  objectives  are  (1)  to  develop  a  variety  of  performance 
measures  needed  to  supplement  those  discovered  in  ongoing  research  or  operational 
programs  or  in  existing  data  bases,  (2)  to  collect  performance  data  in  the  Fleet,  and  (3)  to 
compare  the  various  performance  measures. 

Another  task  in  this  area  addresses  the  development  of  improved  methods  for 
measuring  potential  ability.  In  particular,  this  involves  the  capability  of  new  computer¬ 
ized  measures  to  predict  job  performance.  Tests  of  spatial  abilities  represent  some  of  the 
oldest  selection  measures  in  use.  Paper-and-pencil  tests  have  ignored  several  important 
dimensions  of  dynamic,  or  moving,  spatial  ability  that  can  now  be  tested  using  computer 
presentation.  Speed  of  information  processing,  measured  by  reaction  time,  is  another 
ability  that  was  untestable  in  the  past  but  can  now  be  tested  using  computers. 

Revision  of  Standards 


A  major  task  in  this  area  will  examine  the  current  CLASP  for  potential  to 
incorporate  a  prediction  of  future  job  performance.  The  ultimate  goal  of  this  task  is  a 
new  component  for  CLASP  that  is  based  upon  the  relationships  between  predictors  and  job 
performance. 

This  research  area  also  includes  evaluating  any  proposed  changes  to  CLASP.  One  of 
the  ways  to  evaluate  system  changes  is  in  terms  of  costs  calculated  within  a  utility 
context.  Another  way  is  to  pilot  test  changes  in  standards  through  simulations. 

Detailed  Planning 

Exploratory  development  funds  were  reprogrammed  in  FY83  to  support  additional 
planning  for  JPM  research  to  support  personnel  selection  and  to  start  some  front-end 
analyses.  The  Navy's  detailed  plan  was  presented  early  in  FY83  at  the  joint  NAS 
Committee/JPM  Working  Group  workshop  on  3PM  (Laabs,  1983a).  Additional  planning  was 
reported  to  the  NAS  Committee  in  the  form  of  an  update  later  in  the  fiscal  year  (Laabs, 
1984).  A  systematic  examination  of  alternative  sources  of  job  performance  information 
was  initiated  concurrent  with  the  planning  of  the  Navy's  initial  effort  (Kidder,  Nerison,  St 
Laabs,  1987). 

Since  a  major  Navy  goal  is  to  improve  classification  and  assignment,  the  CLASP 
system  provided  the  context  for  detailed  research  planning  (Laabs,  1983b).  The  roadmap 
for  JPM  research  guided  the  design  of  a  specific  research  project  on  performance-based 
personnel  classification,  which  was  initiated  as  an  advanced  development  project  in  FY84. 


THE  INITIAL  NAVY  PROGRAM: 

PERFORMANCE-BASED  PERSONNEL  CLASSIFICATION 

As  discussed  earlier,  the  Navy's  automated  CLASP  system  is  the  essential  link 
between  the  development  of  standards  and  the  selection  of  personnel  according  to  those 
standards.  It  provides  the  main  structure  for  linking  enlistment  standards  to  job 
performance,  which  is  the  goal  of  the  Joint-Service  Project. 
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The  CLASP  system  has  several  algorithms  for  matching  the  person  to  the  job  within 
constraints  of  fluctuating  recruit  quality  and  occupational  requirements.  It  is  a  real-time 
conversational  computer  program  that  provides  classifiers  with  a  list  of  ratings  appropri¬ 
ate  for  the  recruit  being  processed. 

Data  describing  a  recruit  enter  the  CLASP  system  and  produce  an  optimality  index 
value  reflecting  the  match  between  the  recruit  and  all  ratings.  A  subset  of  ratings  with 
the  highest  values  is  then  produced.  The  recruit  and  classifier  concentrate  on  this  list  of 
ratings,  ensuring  that  recruits  are  assigned  in  a  near  optimum  manner. 

The  optimality  index  is  currently  based  on  six  factors:  (1)  the  match  between  the 
manning  priorities  of  the  Navy  and  the  preferences  of  the  recruit,  (2)  the  match  between 
recruit  aptitude  and  the  complexity  of  the  job,  (3)  minority  representation  across  the 
ratings,  (4)  quotas  for  seats  in  the  various  "A"  schools,  (5)  ASVAB  prediction  of  success  in 
"A"  school,  and  (6)  the  prediction  of  attrition. 


In  order  to  incorporate  a  prediction  of  job  performance  into  CLASP,  the  Navy  needs 
the  capability  to  economically  measure  job  performance  on  a  routine  basis.  This 
requirement  provides  the  orientation  for  the  Performance-Based  Personnel  Classification 
Project. 

The  Navy's  Contribution  to  the  Joint-Service  Project 


The  Navy  launched  "Performance-Based  Personnel  Classification"  as  an  advanced 
development  effort  in  FY84.  The  main  purpose  is  to  investigate  the  feasibility  of 
incorporating  a  performance-based  factor  into  the  current  CLASP  system.  This  is  a 
narrower  purpose  than  setting  minimum  standards  for  service  entry,  which  is  a  long-term 
goal  of  the  Joint-Service  Project.  Thus,  the  emphasis  is  on  establishing  relationships 
between  predictors  and  various  measures  of  job  performance  so  that  a  recruit's  potential 
for  successful  on-the-job  performance  ultimately  can  be  given  formal  consideration  in 
personnel  classification  procedures. 


Overall  Navy  Goals 

The  major  goal  of  the  Navy  effort  is  to  design  and  develop  reliable  and  valid  job 
performance  measures  and  demonstrate  these  measures  as  criteria  for  predictor  valida¬ 
tion.  Laying  the  foundation  for  further  research  to  improve  the  measurement  of  aptitude 
and  job  performance  can  be  considered  a  secondary  goal.  The  design,  development,  and 
fielding  of  the  job  performance  tests,  however,  directly  support  the  Joint-Service  Project 
objective  to  develop  better  methods  and  procedures  for  constructing  hands-on  perfor¬ 
mance  tests  and  to  establish  the  relationships  between  ASVAB  and  job  performance.  It  is 
important  to  note  that  the  Joint-Service  Project  represents  the  first  effort  to  validate 
ASVAB  against  direct  performance  on  the  job. 


PERFORMANCE-BASED  PERSONNEL 
CLASSIFICATION 

•  PURPOSE: 

Investigate  the  feasibility  of 
incorporating  a  performance- 
based  factor  into  CLASP. 

•  TASKS: 

Develop  prototype  job 
performance  measurement 
information  system. 

Identify  six  candidate  ratings. 

Design,  develop,  and  field  job 
performance  test  and  instruments. 

Investigate  feasibility  of  a 
performance-based  component 
for  CLASP. 

Demonstrate  potential  usefulness 
of  new  predictors  (e  g.,  new 
computerized  tests). 

Navy  Research  Strategy 

For  this  initial  effort,  the  Navy  has  limited  the  prediction  of  job  performance  to 
first-term  personnel  in  three  ratings  that  maintain  and  operate  systems  and  equipment 
vital  to  mission  accomplishment.  Achievement  of  the  best  possible  person-job  match  for 
such  technical  ratings  is  crucial  because  of  the  extremely  important  responsibilities  and 
substantial  recruiting  and  training  investment. 


In  accordance  with  the  Joint-Service  research  strategy,  the  Navy  effort  focuses  on 
hands-on  tests  of  technical  proficiency.  Since  such  tests  are  expensive  and  have  to  be 
fielded  on  a  large  scale  to  support  classification  and  assignment  validation,  a  concurrent 
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focus  is  on  economical  substitutes  such  as  using  rating  scales  and  either  paper-and-pencil 
or  computer  simulations  of  job  samples. 

The  research  plan  follows  from  the  roadmap  for  3PM  research  and  includes  three 
major  thrusts  (Laabs,  19S2):  Front-end  analyses  provide  for  the  development  of  a  rating- 
specific  information  system  on  job  performance  and  selection  of  a  preliminary  subset  of 
ratings  for  which  performance  measures  would  be  developed.  Measurement  evaluation 
includes  designing,  developing,  and  fielding  job  performance  measures  (originally  this 
included  personnel-file-based  measures  that  were  subsequently  eliminated)  and  demon¬ 
strating  the  usefulness  of  new  predictors  being  developed  in  related  research  efforts. 
Revision  of  standards  implies  major  operational  changes  in  the  Navy's  CLASP  system  that 
cannot  be  made  as  the  result  of  the  initial  research  work.  Therefore,  this  part  of  the 
research  plan  was  limited  to  an  investigation  of  the  feasibility  of  incorporating  a 
performance-based  factor,  or  component,  into  the  CLASP  system. 

The  five  project  tasks  are  summarized  below: 

1.  Develop  a  prototypic  rating-specific  information  system  to  ensure  that  the 
development  of  job  performance  measures  proceeds  as  economically  as  possible.  Review 
existing  programs,  simulators,  and  data  bases  for  useable  performance  data  or  instruments 
and  for  task  analysis  information  that  could  be  used  in  constructing  performance  tests. 
As  the  project  progresses,  add  rating-specific  information  on  the  predictive  usefulness  of 
various  job  performance  measures. 

2.  Identify  six  candidate  ratings  using  Joint-Service  criteria.  Select  three  ratings 
that  are  critical  to  the  Navy's  mission  and  have  a  large  population. 

3.  Design,  develop,  and  field  job  performance  test  and  instruments.  These  include 
job  sample  tests  and  substitute  measurement  instruments,  consisting  of  job  sample 
simulations  and  behaviorally  anchored  rating  scales.  Common  tasks  are  to  be  included 
across  the  various  tests  and  instruments  in  order  to  evaluate  different  methods  of 
measuring  job  performance. 

4.  Investigate  the  feasibility  of  a  performance-based  factor  or  component  for  the 
CLASP  system.  Evaluate  the  strength  of  the  relationship  between  ASVAB  and  the  various 
job  performance  measures  as  a  basis  for  adding  a  prediction  of  job  performance  to  the 
system.  Evaluate  the  relationships  among  the  different  measures  to  establish  the 
potential  of  each  of  the  substitute  measures  for  routine  gathering  of  job  performance 
data. 


5.  Demonstrate  the  potential  usefulness  of  new  predictors  currently  being  devel¬ 
oped  by  each  service.  Administer  new  computerized  predictor  tests  in  the  Fleet  while  the 
job  performance  data  are  being  gathered.  Establish  the  relationship  among  the  new 
predictors,  the  existing  ASVAB  predictors,  and  job  performance. 

Types  of  Measures 

Hands-on  performance  tests  use  actual  equipment  to  assess  technical  proficiency  and 
may  involve  whole-  or  part-task  sequences.  Factors  considered  in  selecting  tasks  or  part 
tasks  for  inclusion  in  the  test  include  (1)  frequency  of  performance,  (2)  feasibility  of 
testing  in  a  hands-on  mode,  (3)  operational  requirements,  and  (4)  total  testing  time.  The 
hands-on  job  sample  tests  are  needed  as  benchmarks  (i.e.,  standards)  against  which 
substitute  tests  can  be  compared. 


Hands-on  performance  tests  for  critical  tasks  will  be  developed  for  three  ratings. 
Such  tests  are  extremely  expensive  to  construct,  however,  because  of  the  extensive 
analyses  that  must  be  done  of  the  job,  the  job  tasks,  and  the  individual  task  steps  before  a 
valid  and  reliable  test  can  be  produced.  Hands-on  tests  are  even  more  expensive  to 
administer  because  examinees  must  be  observed  with  actual  equipment  in  a  one-on-one 
situation  (usually  for  a  minimum  of  one  workday)  and  examiners  must  not  only  be  job 
experts  but  must  also  be  trained  as  impartial  observers.  Because  it  would  be  prohibitively 
expensive  to  field  hands-on  measures  on  the  large-scale  basis  required  for  validation  work, 
the  Navy  is  investigating  two  types  of  less  expensive  substitute  measures:  simulations  and 
rating  scales  (see  Fig.  2). 


•  BREADTH  OF  COVERAGE 
Narrow 


TECHNICAL  PROFICIENCY 

Assesses  the  capability  to  do  critical  job  tasks. 


GENERAL  JOB  PERFORMANCE 
Assesses  performance  on  job  dimensions 


Broad 

•  DEGREE  OF  FIDELITY 


HANDS-ON  TESTS 

Sample  important  job  tasks  and  use  actual 
equipment  to  test  technical  proficiency. 

JOB  SAMPLE  SIMULATIONS 

Use  paper-and-pencil  techniques  or  training 

equipment  to  test  technical  proficiency. 

RATING  SCALES 

Cover  job  behavior  on  both  specific  technical 
tasks  and  general  job  performance  dimensions. 


Figure  2  Variables  in  Navy  JPM  measures 


One  type  of  substitute  measure  uses  computer-based  simulation  similar  to  that 
frequently  employed  in  training  programs.  Instead  of  high-fidelity  simulations,  in  which 
computers  control  actual  equipment,  this  project  will  use  lower  fidelity  simulations,  such 
as  computer-controlled  videodisc  systems  or  symbolic  simulation.  Videodisc  system 
simulations  present  actual  job  information  (e.g.,  pictures  of  equipment)  and  can  elicit 
responses  related  to  those  required  on  the  job  (e.g.,  touching  a  display  panel  to  change  the 
position  of  a  switch).  Nevertheless,  test  fidelity— the  extent  to  which  the  simulation 
resembles  the  performance  being  tested — is  degraded. 

Symbolic  simulation,  another  form  of  job  sample  simulation,  further  degrades  test 
fidelity  because  it  is  a  paper-and-pencil  test.  Pictures,  diagrams,  and  illustrations 
represent  steps  in  a  job  task;  examinees  answer  multiple-choice  questions  in  reference  to 
the  visual  presentations. 

Another  substitute  for  hands-on  testing  uses  behaviorally  anchored  rating  scales.  The 
information  for  constructing  these  scales  comes  from  the  critical-incident  technique, 
which  identifies  the  behaviors  that  distinguish  poor  performers  from  experts.  Points  on 
these  scales  describe  work  behaviors  associated  with  different  levels  of  task  proficiency. 
The  rating  scales  will  be  developed  at  two  levels:  (1 )  the  specific-task  levels,  a  parallel  to 
the  hands-on  test  and  simulation;  and  (2)  the  performance  dimension  level,  which  refers  to 
broader  job  skills,  such  as  "observing  safety  precautions,"  that  cut  across  specific  tasks. 

Test  Construction  Strategy 

The  same  general  procedure  will  be  used  to  develop  all  three  types  of  measures  for 
each  rating  under  study.  The  steps  involved  are  (1 )  identifying  a  set  of  critical  tasks  that 
adequately  represent  the  job  of  the  first-term  enlistee;  (2)  designing  a  preliminary  job 
sample  by  documentation  of  the  task  to  be  observed  and  identifying  behavioral  steps  to  be 
included  in  the  hands-on  test;  (3)  developing  simulation  tests  using  either  paper-and- 
pencil,  computer-based  simulation,  or  both;  (4)  developing  behaviorally  anchored  rating 
scales  for  specific  tasks  and  for  performance  dimensions;  (5)  trying  out  individual  tests 
and  instruments,  refining  them,  and  establishing  scoring  procedures;  (6)  pretesting  the 
entire  field-test  package  on  a  small  sample  and  training  the  test  administrators;  (7) 
administrating  the  hands-on  test,  simulation,  and  rating  scales  to  a  sample  of  300  first- 
term  enlistees;  and  (8)  analyzing  the  job  performance  data  to  see  if  the  tests  and 
instruments  can  be  substituted,  one  for  another,  and  how  the  performance  data  are 
related  to  predictor-test  data. 

Pretesting  Method 

Two  types  of  pretesting  will  be  done  in  this  project.  One  will  involve  the  preliminary 
tryout  and  refinement  of  the  individual  tests  and  instruments;  the  other,  the  tryout  of  the 
entire  field-test  package.  Refinement  of  the  hands-on  job  sample  test  will  add  and  delete 
tasks  and  part-tasks  to  yield  a  final  test  that  requires  no  more  than  4  to  6  hours  to 
administer.  The  content  of  the  simulation  and  rating  scales  will  match  the  hands-on  test, 
with  refinement  limited  to  format  revisions.  Administration  procedures  will  be  standard¬ 
ized  and  test  observers  will  be  trained  to  ensure  objective  scoring  of  the  hands-on  test. 
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sampling  Approach 

The  following  sampling  design  issues  will  be  considered  during  the  development  of  an 
individualized  plan  for  each  rating: 

1.  The  target  sample  size  adopted  by  the  3oint-Service  Project  is  300  first-term 
enlistees  in  each  rating.  The  attainment  of  this  goal  may  be  limited  by  the  number  of 
ships  or  other  units  available  for  testing. 

2.  For  each  ship  or  unit,  all  first-term  enlistees  who  are  working  in  their  rating  and 
who  have  served  up  to  4  years  in  the  Navy  will  be  tested.  Because  of  informal  job 
specialization,  the  sample  may  have  to  be  restricted  to  a  subset  of  these  first  termers. 

3.  Information  on  race,  sex,  age,  and  education  will  be  collected  routinely  for  all 
examinees.  The  makeup  of  most  of  the  technical  ratings  selected  for  study,  however,  will 
not  accommodate  the  stratification  of  the  sample  on  these  variables. 

Data  Collection 

For  most  ratings,  the  field-test  package  will  be  administered  on  board  ship  and  in 
pier-side  buildings  or  vans.  The  Navy  will  experience  particularly  difficult  logistic 
problems  in  collecting  hands-on  performance  data  because  sailors  are  widely  dispersed 
among  small  and  mobile  units. 

The  total  time  required  to  test  one  examinee  will  be  about  10  hours.  The  total  time 
available  per  examinee  will  be  divided  among  (1)  a  hands-on  job  sample  test,  taking  from  4 
to  6  hours;  (2)  a  job  sample  simulation,  taking  from  1  to  2  hours;  (3)  a  new  predictor  test, 
taking  about  1  hour;  and  (4)  sets  of  rating  scales,  taking  about  1  hour.  The  simulation  and 
self-ratings  will  be  administered  at  the  beginning  of  the  testing  period.  At  the  same  time, 
a  set  of  rating  scales  will  be  administered  to  at  least  two  peers  and  one  first-line 
supervisor  of  each  examinee. 

Additional  information,  such  as  time  on  job  task  and  personnel  record  entries,  will  be 
collected  about  each  examinee.  The  collection  of  these  data  will  require  extensive  liaison 
and  coordination  with  each  ship  that  participates.  Periodically,  a  representative  of 
NAVPERSRANDCEN  who  has  completed  the  test  observer  training  will  act  as  a  second 
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observer  to  ensure  that  testing  continues  in  the  field  in  accordance  with  training  and  to 
reinforce  proper  data  collection  procedures. 

Data  Analyses 


Hands-on  performance  tests  raise  unique  problems  in  evaluating  test  reliability. 
Since  such  tests  are  typically  scored  by  observers,  reliability  depends  on  the  objectivity  of 
their  observations.  To  identify  and  reduce  observer  subjectivity,  test  scoring  procedures 
will  be  standardized  and  observers  will  be  trained.  During  the  pretest,  two  observers  will 
be  placed  at  each  testing  station.  Test  objectivity  will  be  determined  by  estimating  the 
degree  of  agreement  among  observers  across  examinees  and  tasks.  In  addition,  the  data 
from  the  pretest  will  be  analyzed  using  generalizability  theory  designs  to  estimate  the 
variance  associated  with  observers  and  other  facets  that  may  affect  performance  scores. 

The  validities  of  the  hands-on  performance  test  and  its  substitutes  depend  on  the 
tasks  included  and  how  well  these  tasks  represent  the  job  being  studied.  The  adequacy  and 
completeness  of  the  job  content  representation,  or  "content  validity,"  will  be  evaluated 
during  test  development. 

To  further  evaluate  the  tests  and  instruments,  the  following  analyses  will  be 
conducted  on  the  data  collected  from  the  entire  sample:  (1)  examination  of  the 
distribution  of  scores;  (2)  item  analyses;  (3)  interobserver  reliability  using  the  data  from  a 
Navy  representative  as  the  second  observer;  (4)  interrater  reliability  for  peer  ratings;  (5) 
interrater  reliability  across  self-,  supervisor-,  and  peer-rating  sources;  and  (6)  multitrait- 
multimethod  analyses  for  multiobserver  data,  ratings  from  different  sources,  and  the 
three  different  measurement  methods.  A  final  series  of  analyses  will  use  generalizability 
theory  designs. 

An  additional  evaluation  problem  is  associated  with  establishing  th?  rough  equiva¬ 
lence  between  the  hands-on  performance  test  and  the  corresponding  simulation  and  set  of 
rating  scales.  Regression  analyses  will  be  used  to  relate  performance  scores  to  ASVAB 
subtest  scores.  The  predicted  criterion  scores  for  the  three  different  types  of  proficiency 
measures  will  be  compared  by  correlating  their  ranking  of  the  examinees  to  see  whether 
the  tests  and  instruments  are  similarly  ordering  individuals. 


THE  NAVY'S  JPM  PROGRAM 

The  Navy's  Performance-Based  Personnel  Classification  Project  has  expanded  and 
evolved  into  the  wider-ranging  JPM  program,  as  indicated  in  the  latest  annual  report  to 
Congress  (OASD,  Force  Management  and  Personnel;  FMicP;  1985,  Ch.  5).  Increased  Navy 
and  Joint-Service  interest  has  expanded  the  number  of  ratings  being  covered  in  the 
project  and  resulted  in  the  spinoff  of  two  research  tasks  (i.e.,  development  of  a  prototypic 
JPM  data  base  and  demonstration  of  new  predictors)  into  separate  projects  under  the  JPM 
program. 

These  actions  were  stimulated  by  congressional  concern  for  progress  in  the  JPM  area, 
as  evidenced  by  letters  to  OASD  from  the  Chairman  of  the  House  Armed  Services 
Committee  in  1983  and  1985  (App.  C).  These  letters  and  a  subsequent  DoD  memorandum 
(App.  D)  indicated  that  the  Joint-Service  Project  holds  great  potential  for  achieving 
maximum  performance  of  military  manpower  and  requested  that  plans  for  the  future 
implementation  of  JPM  technology  be  drafted. 


KsgsesA  t  vaiaQflgftr  aams 


j 


L  ‘ 


K-: 


Three  project  areas  currently  make  up  the  Navy's  3PM  Program:  (1)  a  Navy  3PM  data 
base,  (2)  component  rating  projects,  and  (3)  a  performance  prediction  project.  Each 
project  or  set  of  projects  can  be  traced  back  to  the  original  roadmap  for  3PM  research. 
Although  several  of  the  projects  started  as  tasks  in  the  Performance-Based  Personnel 
Classification  Project,  they  now  have  independent  funding  status. 

Navy  3PM  Data  Base 

The  Performance-Based  Personnel  Classification  Project  initiated  development  of  a 
prototypic  information  system  containing  3PM  and  prediction  information.  This  system 
will  contain  references  to  available  performance  measurement  instruments  and  devices, 
task  analyses,  and  previously  conducted  performance  measurement  and  prediction  studies. 
One  of  the  main  purposes  of  gathering  this  information  was  to  ensure  that  the  Navy 
proceeds  as  economically  as  possible  in  developing  job  performance  measures. 

The  information  system  will  also  include  the  job  performance  data  collected  under 
the  project.  These  data,  along  with  the  other  performance-related  information,  provide 
the  essential  structure  for  a  3PM  data  base.  If  fully  developed,  the  data  base  can  be  used 
by  the  research  community  to  guide  criterion  development  and  by  the  operational 
community  to  answer  policy  questions  and  guide  decisions.  Because  of  the  substantial 
interest  in  the  3PM  data  base,  the  research  task  was  significantly  increased  in  scope  and 
became  a  separate  project. 

A  basic  component  for  the  data  base  was  constructed  at  the  Navy  Training  Systems 
Center.  This  work,  supported  in  part  by  3oint-Service  funds  from  the  Training  and  Data 
Performance  Center,  evaluated  a  selected  set  of  Navy  training  simulators  as  potential 
sources  of  job  performance  information.  A  computer  data  base  and  catalogue  listing  of 
apprentice-level  Navy  training  devices  has  been  completed  (Morris,  Best,  Sc  McDaniel, 
1985). 


The  Federal  Computer  Performance  Evaluation  and  Simulation  Center  (FEDSIM)  is 
currently  developing  a  life-cycle  automation  management  plan  for  the  3PM  data  base 
(Grizzle  <5c  Kenneth,  1986).  FEDSIM  is  currently  conducting  needs  analysis  that  will 
determine  requirements  for  the  3PM  data  base  in  Navy  research  and  operational 
communities  and  propose  a  framework  within  which  specific  plans,  policies,  and  pro¬ 
cedures  can  be  identified.  The  results  of  the  needs  analysis  will  guide  the  design  and 
construction  of  a  rating-specific  prototype  in  the  outyears  of  the  project. 


Component  Ratings  Projects 

Improved  financial  support  for  Navy  3PM  research  has  expanded  the  number  of 
ratings  that  can  be  studied  in  the  Performance-Based  Personnel  Classification  Project. 
Originally,  performance  measures  were  to  be  developed  for  only  three  of  the  six  candidate 
ratings  identified.  Subsequently,  an  additional  rating  was  added  with  the  aid  of 
supplemental  funding,  and  the  last  two  ratings  were  added  with  an  increase  in  Navy  funds. 
The  six  ratings  now  included  in  the  project  are: 


1. 

2. 

3. 

4. 


Machinist's  mate  (MM) 
Radioman  (RM) 

Electronics  technician  (ET) 
Operations  specialist  (OS) 
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5.  Fire  controlman  (FC) 

6.  Electrician's  mate  (EM) 

The  following  rating  was  selected  for  a  special  Joint-Service  Project  demonstration: 

7.  Aviation  machinist's  mate  (AD) 

The  AD  demonstration,  which  is  primarily  supported  with  Navy  funds,  will  apply  Air 
Force  job  sample  tests  and  substitutes  to  the  corresponding  Navy  and  Marine  Corps  jobs. 

The  seven  ratings  selected  fit  well  in  the  Joint-Service  effort  because  together  they 
cover  about  25  percent  of  the  total  Navy  enlisted  force,  come  from  the  top  26  critical 
Navy  ratings,  represent  5  of  the  24  officially  recognized  occupational  fields,  and  use  5  of 
the  current  10  ASVAB  composites  that  are  in  operation  in  the  Navy's  classification  and 
assignment  system. 

The  Joint-Service  research  strategy  will  be  applied  to  these  seven  ratings—each  of 
which  is  really  a  project  in  itself.  After  the  seven  projects  are  completed,  there  should  be 
sufficient  evidence  to  continue  validation  research  that  uses  one  of  the  substitute 
measures,  thereby  eliminating  more  testing  with  the  expensive  hands-on  measures.  The 
complex  nature  of  hands-on  testing  is  well  illustrated  by  the  job  descriptions  of  the  seven 
ratings  being  investigated: 

Machinist's  mate  (MM):  Operate,  perform  organizational  and  intermediate  level  main¬ 
tenance  on,  and  repair  ship  propulsion  machinery,  auxiliary  equipment,  and  outside 
machinery,  such  as  steering  engine,  windlasses,  hoisting  machinery,  elevators,  food 
preparation  equipment,  laundry  equipment,  and  refrigeration  and  air  conditioning  equip¬ 
ment.  Transfer  and  inventory  lubricating  oils.  Maintain  records  and  reports.  May 
perform  duties  in  the  generation  and  stowage  of  industrial  gases. 

Electrician's  mate  (EM):  Stand  watch  on  generators,  switchboards,  control  equipment,  and 
electrical  equipment.  Operate  and  perform  organizational  and  intermediate  maintenance 
on  power  and  lighting  circuits,  electrical  fixtures,  film  projectors,  motors,  generators, 
voltage  and  frequency  regulators,  controllers,  distribution  switchboards,  and  other 
electrical  equipment.  Test  for  short  circuits,  ground,  or  other  casualties.  Rebuild 
electrical  equipment,  including  solid  state  circuitry  elements,  in  an  electrical  shop. 

Radioman  (RM):  Transmit,  receive,  and  process  all  forms  of  telecommunications  through 
various  transmission  media,  applying  the  basic  principles  of  reliability,  security,  and  speed 
in  accordance  with  appropriate  doctrinal  and  procedural  publications.  Operate,  monitor, 
and  control  telecommunications  transmission,  reception,  terminal,  and  processing  equip¬ 
ment.  Employ  knowledge  of  electronic  and  operational  system  theory  in  applying 
diagnostic  and  restoral  techniques.  Perform  assigned  mission  organizational  level 
maintenance  on  telecommunications  equipment  and  systems. 

Electronics  technician  (ET):  Perform  maintenance  on  electronic  equipment  used  for 
communication,  detection,  tracking,  recognition,  and  identification,  and  on  aids  to 
navigation.  (Exceptions:  airborne  equipment,  data  processing  systems,  interior  communi¬ 
cations  systems,  teletypewriters,  sonar,  dead  reckoning  analyzer  indicators,  weapons 
control  systems,  and  electronic  warfare  systems.) 

Operations  specialist  (OS):  Function  as  plotters,  radio-telephone  and  Command  and 
Control  sound-powered  telephone  talkers,  and  maintain  Combat  Information  Center  (CIC) 


displays  of  strategic  and  tactical  information.  Operate  surveillance  and  altitude  radars, 
Identification  Friend  or  Foe  (IFF),  and  associated  equipment.  Serve  as  watch  supervisors 
and  section  leaders.  Interpret  and  evaluate  presentations  and  tactical  situations  and  make 
recommendations  to  superiors  during  watch  conditions.  Apply  a  thorough  knowledge  of 
doctrine  and  procedures  applicable  to  CIC  operations  contained  in  U.S.  Navy  instructions 
and  Allied  or  U.S.  Navy  publications  and  procedures  necessary  for  radar  navigation 
contained  in  Naval  Oceanograpic  Office  publications.  Provide  command  with  technical 
information  and  assistance  related  to  Anti-Surface  Warfare,  Amphibious  Warfare,  Mine 
Warfare,  Naval  Gunfire  Support,  and  search  and  rescue  operations.  Plan,  organize,  direct, 
and  administer  shipboard  training  programs.  Provide  technical  information  and  advice  on 
capabilities,  limitations,  reliability,  and  operational  readiness  of  CIC  equipment.  Advise 
staffs  and  commands  on  matters  of  operational  readiness,  control  and  use  of  equipment 
and  personnel,  and  other  matters  pertaining  to  the  operations  specialist's  area. 

Fire  controlman  (FC):  Perform  organizational  and  intermediate  level  maintenance  on 
surface  missile  and  gun  fire  control  systems  and  associated  test  equipment.  Operate  and 
test  surface  missile  and  gun  fire  control  systems.  Make  detailed  mechanical,  electrical, 
and  electronic  casualty  analyses  and  operate  associated  test  equipment. 

Aviation  machinist's  mate  (AD):  Maintain  aircraft  engines  and  their  related  systems, 
including  the  induction,  cooling,  fuel,  oil,  compression,  combustion,  turbine,  airborne  gas 
turbine  compressors,  exhaust  and  propeller  systems,  and  preflight  aircraft.  Conduct 
periodic  inspections  on  jet  engines  and  engine-related  systems.  Field-test  and  adjust 
components  of  the  engine  including  fuel  controls,  pumps,  valves,  and  regulators.  Remove, 
repair,  and  replace  compressor  and  turbine  blades  and  combustion  chamber  liners. 
Preserve  and  depreserve  jet  engines,  engine  accessories,  and  components.  Supervise  jet 
engine  work  centers. 

Performance  Prediction 

This  work  will  field  new  measures  of  ability  in  order  to  demonstrate  their  usefulness 
as  new  predictors  for  personnel  selection  and  classification.  Originally,  this  work  was  to 
be  conducted  late  in  the  life  of  the  project,  but  new  test  development  technology  is 
progressing  at  a  very  fast  rate.  An  increasing  amount  of  interest  is  being  shown  in  the 
development  of  new  predictors  to  supplement  ASVAB,  especially  with  the  impending 
advent  of  the  computerized  adaptive  testing  (CAT /ASVAB)  system.  With  the  aid  of  some 
Joint-Service  funding,  this  task  was  significantly  increased  in  scope  and  became  a 
separate  project. 

The  research  strategy  of  this  project  is  to  bring  new,  computerized  ability  tests  into 
the  field  where  performance  data  are  being  collected.  The  research  has  four  major  steps: 

1.  Examination  and  selection  of  new  ability  tests  being  investigated  in  each  of  the 

armed  services. 

2.  Pilot  administration  of  a  new  predictor  in  the  field. 

3.  Adaptation  of  the  selected  new  tests  to  the  computer  system  specified  by  the 

Joint-Service  CAT/ASVAB  Project. 

4.  Demonstration  of  the  test  battery  in  the  field  across  several  different  jobs. 


The  Future  of  3PM  Technolos 


In  a  1985  memorandum  to  the  assistant  service  secretaries,  the  Assistant  Secretary 
of  Defense  (MI&L)  requested  a  plan  for  the  development  of  3PM  technology  (Appendix  D). 
This  plan  was  to  include  a  discussion  of  (1 )  a  least-cost  and  a  highest-cost  option  for 
extending  performance  measurement  to  the  total  enlisted  force,  (2)  the  transition  of  3PM 
technology  to  operational  commands,  (3)  the  creation  of  a  3PM  data  base,  and  (4)  the 
application  of  3PM  technology  to  areas  besides  personnel  selections. 


FUTURE  PLANS  FOR  JPM  TECHNOLOGY 

•  Extension  of  JPM  technology  to  total  enlisted 
force 

•  Transition  of  JPM  technology  to  operational 
commands 

•  Application  of  JPM  technology  to  areas 
besides  personnel  selection 


These  topics  are  discussed  below,  based  on  the  outline  of  plans  that  was  included  in 
the  Fourth  Annual  Report  to  Congress  (OASD,  FMicP,  1985,  Ch.  5).  These  plans  provide  a 
context  for  putting  job  performance  data  from  the  Navy's  3PM  program  into  a  form  that 
can  be  used  by  a  variety  of  personnel  decision  makers. 

Extension  to  the  Total  Enlisted  Force 

Detailed  plans  for  extending  3PM  technology  to  the  total  Navy  enlisted  force  cannot 
be  finalized  until  a  successful  link  to  enlistment  standards  has  been  demonstrated. 
Therefore,  initial  implementation  plans  must  be  based  on  several  assumptions. 

The  first  is  that  the  Navy's  part  of  the  3oint-Service  Project  will  demonstrate  a 
reliable  and  valid  substitute  for  hands-on  performance  measures:  that  is,  rating  scales  (the 
least-cost  option)  or  computer-based  simulations  (the  highest-cost  option).  The  second 
assumption  is  that  the  effort  establishes  significant  practical  relationships  between 
ASVAB  scores  and  job  performance  data  obtained  from  each  type  of  instrument  in  the  two 
implementation  options. 

It  is  further  assumed  that  some  clustering  of  ratings  into  job  families  will  be 
necessary  to  cover  the  total  Navy  enlisted  force  even  if  adequate  surrogates  for  hands-on 
tests  are  found.  Testing  hands-on  to  gather  criterion  data  on  a  routine  basis  is 
prohibitively  expensive  and  time  consuming.  It  is  not  to  be  considered  as  an  option. 

Basic  implementation  of  extension  to  the  total  force  will  require  the  collection  and 
analysis  of  performance  data  from  at  least  two  ratings  from  each  job  family  open  to 
apprentices  (including  the  seven  being  covered  in  the  3oint-Service  Project).  The 
relationships  between  ASVAB  scores  and  job  performance  data  will  be  used  to  incorporate 
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a  prediction  of  job  performance  into  CLASP.  The  changes  to  CLASP  will  be  in  the  form 
of  either  cutoff  scores  or  another  utility  factor. 

The  Navy  has  established  a  formal  evaluation  and  implementation  plan  to  guide  the 
extension  of  JPM  to  its  total  enlisted  force.  The  following  evaluation  events  make  up  this 
initial  implementation: 

1.  After  four  component  ratings  are  completed,  a  simulation  study  will  develop 
cutoff  scores  and  apply  them  to  the  ordered  lists  produced  by  CLASP.  This  will  document 
the  effects  on  the  classification  and  assignment  process  and  generate  information  on  user 
acceptability. 

2.  A  cost-benefit  analysis  of  substitute  measures,  again  l  .er  four  ratings  are 
completed,  will  examine  the  use  of  the  substitutes  on  both  a  one-time  and  recurring  basis. 
The  analysis  will  also  examine  multiple  uses  of  the  performance  data. 

3.  A  systems  analysis  of  alternative  CLASP  implementations  will  be  conducted  at 
the  end  of  the  Performance-Based  Personnel  Classification  Project.  It  will  outline 
techniques  for  clustering  ratings  and  examine  these  alternatives:  (a)  cutoff  scores  for 
seven  ratings  only,  (b)  cutoff  scores  for  "clusters"  of  ratings,  (c)  cutoff  scores  for  all  96 
ratings  in  CLASP,  (d)  a  CLASP  factor  for  clusters  of  ratings,  and  (e)  a  CLASP  factor  for 
all  96  ratings. 

The  tentative  sponsors  for  the  initial  implementation  are  (1)  the  Joint-Service  MAP 
Steering  Committee,  which  will  make  the  service-wide  implementation  decisions;  (2) 
Deputy  Chief  of  Naval  Operations  for  Manpower,  Personnel,  and  Training  (OP-01  )f  (3) 
Commander,  Naval  Military  Personnel  Command  (NMPC-48);  and  (4)  Commander,  Navy 
Recruiting  Command  (CNRC-20).  It  is  anticipated  that  the  sponsors  of  the  initial 
implementation  will  also  be  involved  in  developing  a  long-term  JPM  capability. 

Transition  Requirements.  Any  number  of  Navy  operational  commands  could  take  on 
part  or  all  of  the  responsibilities  for  periodically  measuring  job  performance  in  the  Fleet 
if  a  permanent  JPM  system  were  established.  Alternatively,  a  new  Navy  organization 
could  be  established  solely  for  that  purpose.  Final  institutionalization  decisions  will  be 
made  by  the  sponsors. 

As  JPM  technology  moves  from  research  to  operation,  changes  in  the  oversight  of  the 
work  must  occur.  Engineering  research  and  development  funds  will  be  used  to  support  the 
implementation  that  occurs  in  the  early  years  (i.e.,  evaluation  events),  and  operational 
funds  will  support  the  measurement  work  to  be  accomplished  in  the  later  years.  If  a 
permanent  JPM  system  is  established,  operational  funds  will  be  used  exclusively. 

JPM  Data  Base 

As  already  discussed,  a  rating-specific  data  base  is  part  of  the  Navy's  JPM  program. 
The  main  end  products  will  be  the  prototype  data  base  and  a  set  of  plans,  policies,  and 
procedures  for  transferring  data  to  a  central  site  and  for  operationalizing  the  prototype. 

Applications  of  JPM  Technology 


made  up  of  users  and  sponsors  to  oversee  implementation  and  extension  of  3PM 
technology.  This  group  will  recommend  specific  applications  based  on  their  evaluation  of 
the  research  results. 


PAYOFF/BENEFITS 


•  IMPROVE  PERSONNEL  SELECTION 
Predict  performance 

Validate  new  predictors 

•  TECHNOLOGY  CONTRIBUTIONS 

•  OTHER  PERFORMANCE  DATA  USES 
Training  feedback 
Certification 

Diagnosis 

Resource  allocation 


Training  assessment,  diagnostic  testing,  and  certification  are  the  most  likely  addi¬ 
tional  applications  of  3PM  technology.  Therefore,  representatives  from  the  Chief  of 
Naval  Education  and  Training  (Training  Appraisal  Program),  Naval  Sea  Systems  Command 
(Shipyard  Training  and  Certification),  and  the  Navy  Reserve  Forces  (Training  Tracking 
System)  are  included  in  the  group  of  users  and  sponsors.  As  new  application  areas  are 
uncovered,  appropriate  individuals  will  be  assigned  to  the  implementation  planning  group. 


CONCLUSIONS 

One  of  the  main  end  products  of  the  Navy's  3PM  program  will  be  field-tested 
performance  measures  and  relationships  that  make  it  possible  to  predict  job  performance 
within  the  CLASP  system.  The  narrowest  implementation  would  be  the  institution  of 
cutoff  scores  for  the  seven  critical  ratings  included  in  the  Performance-Based  Personnel 
Classification  Project  that  affect  about  25  percent  of  the  Navy  enlisted  population.  The 
broadest  implementation  would  cover  100  percent  of  the  Navy  enlisted  population  by 
developing  a  performance-based  factor  for  CLASP.  This  would  require  additional  data 
collection  and  the  institutionalization  of  3PM  in  the  Navy. 

Another  end  product  of  the  Navy's  3PM  program  will  be  the  advancement  of 
technology.  Easier,  more  economical  means  of  measuring  job  performance  should  lead  to 
more  widespread  use  of  this  important  information  in  a  variety  of  personnel  system 
functions.  The  3PM  program  will  actively  seek  implementation  and  application  opportuni- 


The  field  demonstration  of  new  computerized  tests  that  assess  abilities  that  have  not 
been  measurable  in  the  past  is  another  important  end  product.  The  predictive  relationship 
between  these  tests  and  job  performance  will  provide  needed  information  about  which 
tests,  if  any,  are  candidates  to  supplement  the  computerized  ASVAB. 

A  final  product  of  the  3PM  program  will  be  the  3PM  data  base.  It  will  contain  rating- 
specific  data  on  technical  proficiency  and  general  job  performance  as  well  as  perfor¬ 
mance-related  data  from  past  research  and  other  data  bases,  systems,  and  programs.  This 
information  should  prove  useful  to  the  research  community  in  future  criterion  develop¬ 
ment  efforts,  and  to  the  operation  community  as  they  go  about  making  personnel  and 
resource  allocation  decisions. 

Taken  together,  these  products  will  yield  more  optimal  assignment  into  initial  jobs 
and  an  enhanced  personnel  system.  This  will  lead  to  (1)  improvements  in  individual 
performance,  with  concomitant  improvements  in  readiness;  and  (2)  more  satisfied 
personnel,  with  concomitant  increases  in  morale,  productivity,  and  retention. 
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1980  OFFICE  OF  THE  ASSISTANT  SECRETARY  OF  DEFENSE 
MEMORANDUM  TO  SERVICE  SECRETARIES 


ot.VNF'C'ACR 

Ursi.Hvf.  -'Fairs 
and  logistics 


assistant  secretary  OF  DEFENSE 

WASHINGTON  O  C  J0  30I 


JUL  7  1380 


KEMORAflDUM  FOR  Secretary  of  the  Amy 
Secretary  of  the  Kavy 
Secretary  of  the  Air  Force 

SUBJECT:  Plan  for  Validating  Enlistment  Standards  Against  Job  Performance 

I  believe  the  Department  of  Defense  should  base  its  standards  for  enlistment 
and  for  assignment  to  a  military  specialty  upon  the  probability  of  successful 
job  performance  later  in  a  person's  military  career.  Therefore,  I  am  asking 
the  Services  to  undertake  an  effort,  in  cooperation  with  OSD,  to  establish 
standards  for  enlistment  and  assignment  to  training  that  are  validated  against 
eventual  job  performance.  In  this  context,  validation  of  a  proposed  enlistment 
or  assignment  standard  means  demonstrating  that  enlistees  meeting  it  have  a 
reasonable  probability  of  eventually  attaining  acceptable  levels  of  job  per¬ 
formance. 

The  overall  effort  will  be  divided  into  three  phases;  1)  a  pilot  project  to  dem¬ 
onstrate  the  feasibility  of  setting  standards  based  on  the  above  critera,  2)  a 
long-term  program  to  establish  and  validate  standards,  and  3)  a  long-term  research 
effort  to  improve  DoD's  measures  of  potential  ability  and  job  performance. 

Pilot  Project 

My  staff  will  continue  its  study  of  Army  enlistees  in  order  to  demonstrate  the 
feasibility  of  setting  enlistment  and  assignment  standards  based  on  existing 
performance  standards.  The  effort  will  focus  on  the  military  specialty  of  in¬ 
fantryman.  This  occupation  was  chosen  because  it  is  critical  to  the  Army's  mis¬ 
sion,  has  a  large  number  of  entrants  each  year,  and  has  a  well  developed  Skill 
Qualification  Test  (SQT)  with  over  9,000  test  results  available  for  analysis. 

My  staff  will  consult  with  the  Army  on  this  pilot  project  and  may  ask  for  some 
additional  data.  They  nay  also  visit  some  Army  installations  and  employ  appro¬ 
priate  contractual  support  in  completing  this  task.  I  have  asked  them  to  complete 
the  pilot  project  by  September  30,  1980. 

I  am  also  asking  each  Service  to  begin  a  similar  pilot  project  for  one  military 
occupation.  These  projects  should  be  completed  by  September  30,  1980.  The  in¬ 
dependent  pilot  projects  should  be  coordinated  among  the  Services  and  with  OSD  to 
ensure  an  exchange  of  information  and  mutual  assistance.  Hy  staff  will  periodically 
request  informal  progress  reports  from  the  Services  on  these  piloc  projects. 
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7  have  attached  an  outline  of  how  my  staff  intends  to  approach  this  pilot  project 
an" validation.  1  expect  that  the  final  report  uould  follow  this  outline.  I  sug¬ 
gest  that  the  Services  adopt  this  sane  approach  aiming  toward  a  similar  final  repo: 
this  fall,  and  that  we  work  together  to  refine  this  methodology  during  the  course 
of  ou/pilot  effort.  I  recognize  that  within  the  tine  available  to  do  the  pilot 
project  it  nay  not  be  possible  to  develop  all  the  data  called  for  by  the  outline 
with  the  degree  of  precision  I  would  like.  Our  plan  is  to  use  the  best  estimates  [ 
available ,  and  document  then;  I  encourage  the  Services  to  do  the  same. 

Hr.  Robert  A.  Stone,  Deputy  Assistant  Secretary  of  Defense  (Program  Management) 
will  be  in  charge  of  this  project  within  the  Office  of  the  Secretary  of  Defense, 
and  will  coordinate  the  Service  efforts.  He  will  have  a  meeting  shortly  with  your 
representatives  to  discuss  this  project  and  to  explain  how  the  OSD  work  will  procei 

Long-Term  Effort  to  Validate  Enlistment  Standards  Against  Job  Performance 

Assuming  that  our  pilot  projects  show  that  it  is  possible  to  link  enlistment  and 
assignment  standards  to  job  performance,  the  Services  should  initiate  in  October 
19S0°a  long-term  project  to  do  this.  The  aim  of  the  project  will  be  to  either 
validate,  or  revise  as  necessary,  current  enxistment  and  assignment  standards  for 
at  least  75  percent  of  the  accessions  entering  training.  The  initial  work  should 
concentrate  on  using  current  ASVAE  scores  and  existing  performance  measures.  As 
better  measures  are  developed  through  the  concurrent  research  effort,  the  standards 
should  be  updated  to  improve  our  ability  to  select  people  for  specific  types  of 
training.  Each  analysis  of  a  proposed  standard  for  a  military  occupation  should 
specifically  examine  the  trade-off  between  the  costs  associated  with  failure  rates 
of  lew  aptitude  people  vs.  the  cost  of  recruiting  higher  aptitude  people  in  order 
to  reduce  the  failure  rate.  Each  revised  standard  should  be  submitted  for  ny  ap¬ 
proval  prior  to  implementation.  1  plan  to  develop  a  mechanism,  in  cooperation  with 
the  Services,  to  monitor  the  progress  on  this  long-term  effort.  Hy  goal  is  to  have 
the  effort  completed  by  the  end  of  FY  19S3. 


Lonp-Term  Research  Program 

Both  the  pilot  projects  and  the  long-term  validation  effort  will  of  necessity  deal 
at  first  only  with  existing  tests  and  performance  measures.  However,  the  Department 
needs  to  start  now  to  develop  improved  methods  for  measuring  potential  ability  and 
job  performance.  Therefore,  the  Services  should  plan  to  initiate  or  re-orient  re¬ 
search  work  in  this  general  area  and  insure  that  adequate  resources  are  provided 
to  address  this  problem  starting  in  FY  1981. 

The  Services  should  request  their  personnel  research  laboratories  to  begin  research 
orojects  for  specific  occupations.  This  research  would  then  become  input  to  the 
long-term  effort  to  validate  standards.  The  initial  phase  of  the  research  effort, 
in  addition  to  providing  the  underpinning  for  future  research,  would  be  in  support 
of  the  skills  selected  for  validation  in  the  long-term  effort. 
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1983  MANPOWER  ACCESSION  POLICY  STEERING  COMMITTEE  ENDORSEMENT 
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HUMAN  RESOURCES  NEED 


1.  TITLE:  Joint-Service  Effort*  to  Link  Standard*  for  Enlistment  to  On-The-Job  Performa 

2.  STATEMENT  OF  NEED:  The  Services,  in  cooperation  with  DoD,  have  undertaken  an 
effort  to  develop  Job  performance  measures  for  individual*  and  Army  units  and  to 
test  the  feasibility  of  linking  enlistment  standards  to  Job  performance  Information. 

In  order  to  achieve  these  goals.  Job  performance  measures  must  be  developed. 

Accordingly,  the  Manpower  and  Accession  Policy  Steering  Committee  (MAPS)  endorses 
the  following  research  game  plan  and  description  of  Service  responsibilities  and 
urges  Service  support  of  research  funding  requirements  and  field  test  data  acquisition 
activities.  All  timelines  are  based  on  the  availability  of  projected  funding  and 
resources.  The  Project  game  plan  can  be  summarized  as  follows: 

a.  The  Services  will  identify  a  set  of  cross-Service  and  Service— specific 
specialties  (MOSs,  APSs,  Ratings)  which  will  serve  as  the  initial  test  bed  for  the 
development  of  Job  performance  measurement  strategies.  (Timeline:  to  be  completed 
by  June  1983) 

b.  Service-specific  funds  will  be  used  to  establish  a  set  of  hands-on  and  6urrogat 
measures.  (Timeline:  FT  1983-1986) 

1)  It  is  projected  that  the  first  set  of  hands-on  and  surrogate  tests  will 
be  ready  for  demonstration  in  FT  1984. 

2)  Each  Service  will  develop  specialized  expertise  in  the  following  particular 
aspects  of  the  Joint-service  performance  measurement  development  effort  and  agrees  to 
share  whatever  techniques  are  found  to  be  most  valid  and  efficient.  (Timeline:  results 
for  initial  skills  in  FT  1985) 

a)  Army:  Army-wide  performance  measures,  MOS-spedfic  job  knowledge  paper 
and-pencil  tests,  unit  performance. 

b)  Air  Force:  Walk-through  testing  procedures,  ratings  of  Job  experience. 

c)  Navy:  Simulator  and  training  device  performance  tests,  symbolic  sim¬ 
ulation  substitute  tests. 

d)  Marine  Corps:  Identifying  and  monitoring  peripheral  data  collection. 

3)  In  addition,  each  Service  will  act  as  the  'lead  Service'  in  demonstrating 
Job  performance  measures  for  one  cross-Service  military  specialty.  (Timeline:  data 
collection  complete  for  first  set  in  FT  1985). 

c.  Joint-Service  6.4  funds  (FE  64722A  and  FE  64709N)  may  be  used  by  the  Services 
to  support  fielding  and  demonstrating  the  administration  of  prototype  hands-on  and 
surrogate  tests  and  data  analyses.  In  addition,  individual  performance  measurement 
development  efforts  will  be  coordinated  and  cross-walked  with  Joist-Service  activities 
evaluating  Army  unit  performance  using  instrumented  ranges.  (Timeline:  initiate  in  FY 
1984,  complete  in  FT  1986) 
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d.  Appropriate  Servl ce-Bpecl f 1 c  data  banks  will  be  established  within  each  Service 
support  Job  performance  data  collection  and  analysis  efforts.  Within  guidelines  establl 
and  agreed  to  by  the  Services,  this  information  will  be  shared  and  stored  centrally  with 
the  Department  of  Defense.  (Timeline:  initial  phase  complete  FT  1986) 

*  3.  BACKGROUND:  The  Bouse  Armed  Services  Committee  and  the  Bouse  Appropriations  Coomitte 

have  both  endorsed  this  effort.  In  addition,  the  HAC  has  directed  OSD  to  provide  manage 
nent  oversight  and  asked  that  DoD  and  the  Services  submit  an  annual  report  of  progress 
each  December.  The  Services  and  OSD  need  to  provide  adequate  support  to  ensure  the 
Cong ressional  mandate  is  addressed  and  achieved  in  the  time  frame  expected  (i.e.,  testinj 
the  feasibility  of  linking  enlistment  standards  to  job  performance  no  later  than  the  end 
of  FT  1986). 

4.  APPLICATION  OP  RESULTS:  This  is  an  important  technical  undertaking  which  could  signi¬ 
ficantly  improve  the  Services*  ability  to  match  people  and  jobs  more  efficiently,  increai 
technical  training  and  subsequent  job  performance,  and  provide  information  which  could 
facilitate  identification  of  training  requirements  for  new  weapons  systems  and  the  devel¬ 
opment  of  new  training  programs. 


5.  PRIORITY:  1 

6.  POINT  OP  CONTACT:  Lt.  Col.  Charles.  R.  Curran,  OASD(MRASL),  Extension  7-9271. 

7.  ENDOP-SEMENT :  The  undersigned  members  of  the  Manpower  Accession  Policy  Steering  Com¬ 
mittee  endorse  the  game  plan  for  the  Joint-Service  job  performance  measurement  research 
pr  -am  and  encourage  DoD/Service  support  for  funding  and  data  collection  efforts. 


Dr.  G.  Thomas  Sicilia 
Director  of  Accession  Policy 
OASD(KRA&L) 


Rear  Adrflral  A.  J.  Her-berger 
DlrecKy/of  Military  Personnel 
Policy  Division 
Office  of  the  Chief  of  Naval 
Operations 


Major  General  H.  N .  Schwarzkopf 
Director  of  Military  Personnel 
Ja/{/  Management 

HQ,  Department  of  the  Army 


"Hajor  General  K.  L.  Peek,  Jr^ 
Director  of  Personnel  Plans 
HQ,  D.S.  Air  Force 


Major  General  A.  Lukeman 
Director  of  Manpower  Plans  and 
Policy  Division 
BQ,  U.S.  Marine  Corps 
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1983  AND  1985  U.S.  HOUSE  OF  REPRESENTATIVES  COMMITTEE 
ON  ARMED  SERVICES  LETTERS  TO  THE  OFFICE  OF  THE 
ASSISTANT  SECRETARY  OF  DEFENSE 


8 


& 


T8.A s>.  $ou£e  o!  ftEprE$Entatibfg 

COMMITTEE  ON  ARMED  SERVICES 

8EaSfjington,  BC  20515 
February  25,  1985 


Honorable  Laurence  Kbrb 
Assistant  Secretary  of  Defense 
Tbe  Pentagon 
Washington,  D.C.  20301 

Dear  Larry: 

Thank  you  for  sending  me  your  December  1984  report,  "Joint-Service  Efforts  to 
Link  Enlistment  Standards  to  Job  Performance."  As  you  know,  I  have  followed  this 
project  with  interest  and  am  pleased  to  note  the  progress  made. 

After  reviewing  the  report,  I  would  appreciate  answers  to  the  following  questions 
by  April  1. 

1.  Why  is  the  Navy's  funding  for  job  performance  measurement  research  so  low 
relative  to  the  other  services?  Does  the  levy's  funding  somehow  reflect  a  lack  of 
commitment  to  the  project? 

2.  What  are  the  criteria  Defense  will  use  to  determine  if  this  project  is  a 
success  and  should  be  made  operational?  When  will  these  criteria  been  applied  to 
make  a  go/no  go  decision? 

3.  Assuming  the  project  is  successful  for  the  20  percent  of  the  force  being 
considered,  what  are  the  plans  to  cover  the  remaining  80  percent?  Are  there  funds 
programmed  for  the  next  phase? 

4.  What  are  the  Department  of  Defense's  plans  for  applying  this  job  performance 
data  to  areas  of  personnel  management  (e.g. ,  training  course  design  and  evaluation, 
promotion  appraisal,  career  development,  etc.)  other  than  setting  enlistment  standards? 

5.  Is  there  still  a  Navy  Tri -Service  PE  in  this  area?  I  wrote  the  Department 
about  this  very  issue  and  njy  support  for  these  joint  service  R&D  efforts  September  20, 
1983  (See  attachment). 

I  continue  to 
area  that  can  be 
productivity,  and  thus 
the  Congress  must 
holds  great  potential 
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September  20,  1983 


'  »  m 


Honorable  Caspar  W.  Weinberger 
Secretary  of  Defense 
The  Pentagon 
Washington,  D.  C. 

Dear  Mr.  Secretary: 

The  recent  conference  on  the  Cefense  Authorization  Act  for  Fiscal  Year 
1984  eliminated  funding  for  joint-service  personnel  and  training  research 
programs  (Program  Elements  64709N  and  64722A)  which  Congress  had  previously 
endorsed  and  instructed  the  Department  of  Defense  to  conduct  with  all  due 
speed.  I  am  particularly  concerned,  because  two  very  critical  joint  service 
efforts  to  link  enlistment  standards  to  job  performance  and  to  develop  a 
computer  adaptive  testing  system  were  largely  funded  by  these  program  elements. 
Both  of  these  programs,  as  well  as  the  other  manpower  and  training  initiatives 
covered  by  these  funds,  have  tremendous  potential  to  improve  significantly 
the  ability  of  the  services  to  meet  manpower  and  training  objectives  in  an 
era  of  increasing  weapons  sophistication  and  decreasing  youth  cohort  size. 

The  elimination  of  these  funds  was  an  unfortunate  consequence  of  the 
turmoil  and  time  constraints  surrounding  the  conference.  I  would  not  take 
it  as  a  signal  that  those  of  us  who  deal  with  manpower  requirements  are  now 
less  interested  in  these  important  joint-service  personnel  and  training 
projects.  We  remain  concerned  and  interested  in  the  successful  completion 
of  these  projects.  As  a  result,  I  would  respectfully  ask  that  the  Department 
of  Defense  ensure  that  adequate  funding  for  joint-service  personnel  and 
training  programs  be  included  and  fenced  (e.g.,  made  items  of  special  interest) 
in  next  year's  (fiscal  1985)  budget  to  cover  promising  personnel  and  training 
technology  demonstrations,  including  the ^erffTs-C^en t  standards/job  performance 
and  computer  adaptive  testing  programs,  (i  will  vigorously  support  these  funds 
during  next  year's  authorization  activities. 

I  sincerely  appreciate  your  eff/rts  t\  ensure  thatAthese  important 
joint-service  research  projects  appsar  in  ore  fiscal  yelr  1985  Department 
of  Defense  budget.  (  1  J 

S  i  n  c  I 


LA :wnw 
copies  to: 

Honorable  Law'rence  J.  Korb 


Les  fcspin 

Chairman,  Military  Personnel 
and  Compensation  Subcommittee 


THE  ASSISTANT  SECRETARY  OF  DEFENSE 

WASHINGTON  D  C  20301-4000 

6  MAY  1365 
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INSTALLATIONS 
ANO  LOGISTICS 

MEMORANDUM  FOR  ASSISTANT  SECRETARY  OF  TEE  ARMY  (M&RA) 

ASSISTANT  SECRETARY  OF  THE  NAVY  (M&RA) 

ASSISTANT  SECRETARY  OF  THE  AIR  FORCE  (MRA&I) 

SUBJECT:  Joint-Service  Job  Performance  Measurement/Enlistment 

Standards  Project 

In  a  February  25,  1985  letter  (shown  at  Tab  A),  Congressman  Les  Aspln 
asked  several  questions  regarding  future  directions  of  the  Joint-Service 
Job  Performance  Measurement/Enlistment  Standards  Project.  On  April  10, 

1985,  I  responded  (Tab  B)  by  indicating  that  OSD  and  the  Services  would 
work  cooperatively  to  develop  plans  and  funding  proposals  for  applying 
job  performance  measurement  (JPM)  information  to  areas  of  personnel 
management  other  than  setting  enlistment  standards.  The  purpose  of  this 
memorandum  is  to  initiate  actions  related  to  my  commitment  to  Mr.  Aspin, 
and  to  provide  you  with  guidance  for  developing  plans  for  the  future  of 
JFM  technology. 

Accordingly,  I  would  appreciate  your  preparing  a  plan  for  the  development 
of  JPM  technology.  This  plan  should  address  the  following  Issues  and  their 
related  costs:  (1)  extending  JPM  measures  and  enlistment  standard  linkages 
to  the  total  force;  (2)  institutionalizing  JPM  data  development;  (3) 
creating  a  centralized  data  base  for  JPM  information;  and  (4)  exploring 
applications  of  JPM  technology  to  other  manpower  policy  areas. 

Your  plans  should  take  into  account  the  following  guidelines: 

(a)  Although  detailed  plans  for  extending  JPM  technology  to  the  total 
force  cannot  be  finalized  until  successful  linkage  to  enlistment  standards 
has  been  demonstrated,  programming  cycles  demand  that  future  funding 
requirements  be  estimated.  Each  Service  should  assume  that  the  demonstration 
project  is  successful  and,  in  that  light,  develop  both  least-cost  and 
highest-cost  options  that  will  permit  extension  of  performance  measurement  to 
the  rest  of  the  force. 

(b)  Plans  for  transitioning  JPM  technology  beyond  the  research  phase 
should  make  provisions  for  identifying:  organizational  responsibilities  for 
long-term  JPM  development  within  each  Service,  funding  and  policy  changes 
needed  to  make  this  happen,  and  a  timetable  for  key  milestones. 

(c)  Plans  for  creating  a  JPM  data  base  should  indicate  how  and  when 
each  Service  plans  to  establish  common  data  elements  and  formats,  policies 
on  release  and  use  of  data,  and  a  timetable  for  transferring  data  from 
Service  data  bases  to  a  central  Defense  Training  Data  and  Analysis  Center 
data  base. 


(d)  The  Services  are  encouraged  to  consider  any  application  of  JPM 
technology  with  the  potential  for  improving  manpower,  personnel  and  training 
procedures.  Such  areas  as  training  curricula  development  and  evaluation, 
and  reenlistment  standards-setting  would  appear  to  be  among  the  more 
promising  areas.  Plans  should  also  include  proposed  changes  to  the  current 
management  structure  of  the  Joint-Service  Project  to  facilitate  the 
transition  of  current  JPM  technology  to  these  new  areas. 

I  would  appreciate  receiving  your  plans  by  September  1,  1985,  so  as  to  permit 
their  review  and  discussion  at  the  fall  quarterly  meeting  of  the  Joint-Service 
Job  Performance  Measurement  Working  Group.  I  would  also  expect  that  the 
Services  would  include  the  plans  in  their  respective  chapters  of  the  forthcoming 
Fourth  Annual  Report  to  Congress,  scheduled  for  submission  in  December  1985. 
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